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I.  In  the  second  year  (March  15,  1987  -  April  15.  1988)  of  AFOSR 
support,  research  work  on  the  stochastic  behavior  of  the  LMS  and  related 
algorithms  has  yielded  results  in  four  major  areas: 

A.  Data  Normalized  LMS-Transient  behavior  (51 

This  work  generalized  the  efforts  of  the  previous  year  to  the  transient 
behavior  of  arbitrary  linear  filters  for  estimating  the  input  power  level.  The 
transient  mean  performance  of  an  exponentially  weighted  power  estimator 
was  analyzed  and  the  analysis  supported  by  simulations. 

B.  Joint  Probability  .Density  Fqflctions_gL the  LMSJEfiighla  during. 

Adaptation  12.6.1 11 

Significant  progress  has  been  made  in  determining  the  joint  transient  and 
steady-state  probability  density  functions  of  the  time  domain  LMS  weights 
(61  and  the  complei  scalar  frequency  domain  weights  (2,1 11.  It  was  shown  in 
(61  that,  after  an  initial  transient,  the  weights  are  jointly  gaussian  for  small 
values  of  ^  with  time-varying  mean  vector  and  covariance  matrix  given  by 
the  solution  of  the  well-known  difference  equations  for  the  weight  vector 
mean  and  covariance  matrix.  For  the  complex  scalar  LMS  algorithm,  it  was 
shown  in  (2,1 1]  that  the  real  and  imaginary  parts  of  the  fluctuations  about 
the  Wiener  weight  are  jointly  log-normal  distributed  for  small  n  and 
statistically  dependent  gaussian  random  variables  for  large  n. 

C.  Application  of  the  Results  of  B  to  a  Spread  Spectrum  Communications 

Problem  [4,71 

The  effect  of  the  weight  fluctuations  of  the  ALE  on  the  bit  error  rate  of  a 
Direct-Sequence  Spread  Spectrum  Communication  system  has  been 
investigated  (4,7).  The  ALE  is  inserted  in  the  system  in  order  to  reject 
narrowband  interference.  The  converged  ALE  weights  are  modeled  as  the 
parallel  connection  of  a  deterministic  FIR  filter  (Wiener  weights)  and  a 
random  FIR  filter.  The  output  of  the  random  FIR  filter  was  shown  to  be  non- 
gaussian  and  to  significantly  degrade  the  error  probability.  Error  probability 
expressions  were  derived  for  the  bit  error  rate  for  monochromatic  and 
nfcrrowband  sinusoidal  gaussian  jammers  ( 4 J. 

D.  Non-Linear  Effects  in  LMS  Adaptation  (1.3.8-131 

8f 
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The  effects  of  the  various  non-linearities  on  the  behavior  of  the  time  domain 
LMS  algorithm  (1.3,  8-131  have  been  successfully  analyzed: 

1)  It  was  shown  111  that  the  stopping  phenomena  in  quantized  LMS 
adaptation  can  be  removed  by  the  addition  of  spectrally  shaped  dither 
without  significantly  effecting  the  steady-state  mean  square  error  of  the 
quantized  algorithm. 

2)  It  was  shown  (31  that  a  saturation  type  error  non-linearity  in  the 
weight  up-date  equation  causes  a  slow-down  in  the  algorithm  convergence 
rate  directly  proportional  to  the  degree  of  saturation.  Quoting  from  page  23 
of  (31:  "Thus ,  the  trade-off  between  speed  of  convergence  and  saturation 
was  investigated  for  a  filed  steady-state  excess  mean-square-error.  A 
functional  relationship  was  derived  between  the  convergence  rate  of  the 
LMS  algorithm  and  the  number  of  bits  in  a  fixed  point  implementation  of  the 
feedback  error  signal.  This  relationship  was  tabulated  and  shows  that  1 ) 
starting  with  a  sign  detector,  as  one  increases  the  number  of  bits  in  the  error 
signal  representation,  the  convergence  rate  is  increased  by  nearly  a  factor  of 
two  for  each  additional  bit,  2)  as  the  number  of  bits  is  increased  further, 
additional  bits  buy  very  little  in  additional  convergence  speed,  and  3)  as  the 
number  of  bi*s  increase  further,  the  behavior  approaches  that  of  the  linear 
algorithm". 

3)  Analog  implementation  of  the  LMS  and  Block  LMS  algorithms  have 
been  shown  equivalent  with  respect  to  adaptation  speed  and  steady-state 
mis-adjustment  errors.  However,  analysis  and  simulation  of  digital 
implementations  of  the  same  algorithms  have  shown  significantly  different 
behavior  with  respect  to  transient  response  and  algorithm  stalling  18.121  for 
a  filed  number  bits.  Mathematical  models  have  been  derived  which  predict  > 
the  different  behaviors  and  allow  one  to  design  the  appropriate  algorithms.  ( 

4)  The  mathematical  models  derived  in  (3)  have  been  extended  to  the  \| 

non-white  data  case  for  the  LMS  algorithm  configured  as  an  Adaptive  Line 
Enhancer  (9,141.  Algorithm  convergence  slowdown  is  predicted  by  the 
theory.  Weak  signal  suppression  is  also  predicted  by  the  theory.  Both  n  ?or 
predictions  are  supported  by  simulations.  a*i 

5)  The  nonlinear  error  feedback  effects  analyzed  in  13)  have  been 

eitended  to  the  entire  LMS  weight  update  term.  A  (1  -eip(-x)l  nonlinearity  ’f  J 
is  used  to  model  the  effects  of  saturation  (10,131.  The  tradeoff  between 
extent  of  saturation,  algorithm  convergence  speed  and  steady-state  excess 
mean  square  error  are  studied  as  in  (31.  By  comparison  with  (31,  it  ic  :  :o, 

concluded  that  "there  is  no  significant  difference  in  the  behavior  of  digital  m 

implementations  of  the  LMS  algorithm  whether  round-off  occurs  before  or  u 

after  multiplying  the  error  by  the  data".  Jlat  ^p®c 
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These  results  impact  significantly  on  the  usefulness  and  applications 
of  LMS  adaptive  filtering: 

A.  Although  the  LMS  algorithm  is  simple  to  implement,  one  of  its 
significant  drawbacks  is  speed  of  convergence.  The  work  in  [1]  enables  one 
to  design  the  algorithm  so  as  to  obtain  a  fast  response  to  changing  input 
power  levels  with  sufficient  smoothing  for  fixed  unknown  power  levels. 

B.  Although  the  mean  square  error  behavior  of  the  LMS  algorithm  is  well 
understood,  there  are  many  situations  where  additional  statistical 
information  about  the  weight  vectors  would  be  useful.  These  cases  include: 

1 )  detection  of  narrowband  line  component  in  background  noise  using  the 
weight  vector  as  a  test  statistic  (adaptive  line  enhancer)  (6],  2)  using  the 
adaptive  filter  output  as  a  test  statistic,  3)  time  delay  estimation  using  the 
weights  [61. 

C  Recently  the  LMS  algorithm  has  been  imbeded  in  larger  systems  for 
the  purposes  of  enhancing  desired  signals  and  canceling  undesired  signals. 

In  this  mode  of  operation,  it  is  often  necessary  to  know  more  than  just 
second  moment  statistics  of  the  weights.  As  an  example,  the  LMS  canceller 
has  been  used  to  cancel  undesired  narrowband  interference  in  spread- 
spectrum  communication  systems.  [4]  makes  use  of  the  results  in  [6]  to 
study  the  bit  error  probability  of  a  DS  Spread -Spectrum  system  using  an  ALE 
for  narrowband  interference  rejection.  The  effects  of  the  ALE  weight 
misadjustment  errors  on  the  bit  error  rate  are  investigated.  The  converged 
ALE  weights  are  modeled  as  a  parallel  connection  of  a  deterministic  FIR  (the 
Wiener  weights)  and  a  random  FIR  filter  (weight  mis-adjustment  errors) 
with  gaussian  statistics.  Using  properties  of  the  ALE  weight  mis-adjustment 
errors  derived  elsewhere  and  in  16],  error  probability  expressions  are 
derived  for  the  bit  error  rate  for  monochromatic  and  sinusoidal  jammers. 

D.  Digital  implementation  of  the  LMS  algorithm  essentially  involves 
some  non-linear  modifications  of  the  basic  algorithm.  Understanding  the 
effects  of  various  non-linearities  is  important  in  choosing  the  kind  of 
implementable  non-linearities  that  yield  rapid  convergence  rates  and  small 
steady-state  mis-adiust»o*m  errors.  The  theory  developed  in 
[1,3.9,10,13.14]  enables  one  to  better  understand  this  non-linear  behavior. 

When  the  LMS  algorithm  is  implemented  in  block  form  to  take 
advantage  of  the  speed  of  block  processing,  finite  word  effects  are  different 
for  the  LMS  and  BLMS  algorithms.  Using  gausssian  data  models,  [8,12] 
investigated  these  differences  and  showed  that  the  LMS  algorithm  is  less 
sensitive  to  finite  word  effects  than  the  Block  LMS  algorithm. 
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